Partitioning with Space-Filling Curves on the Cubed-Sphere

Numerical methods for solving the systems of partial differential equations arising in geophysical fluid dynamics rely on a variety of spatial discretization schemes (e.g. finite difference, finite element). For parallel execution on distributed memory computers, the computational domain must be partitioned. The choice of partitioning algorithm can have a significant impact on the sustained floating point execution rate of an atmospheric model. The NCAR spectral element atmospheric model employs a gnomonic projection of a cube onto the surface of the sphere. The six cube faces are each subdivided into an array of quadrilateral spectral elements. When the cubed-sphere is partitioned using METIS, both computational load imbalance and communication requirements can lead to sub-optimal performance. In this paper, Hilbert, Peano, and nested Hilbert m-Peano space-filling curves are investigated as the basis of alternative partitioning algorithms. The resulting partitions allow a maximum 22% increase in the sustained floating point execution rate versus METIS on O(1000) processors, when running a relatively high resolution climate simulation.